A multi-modelS based approach for the modelling and the analysis of usable and resilient partly autonomous interactive systems

La croissance prévisionnelle du trafic aérien est telle que les moyens de gestion actuels doivent évoluer et être améliorés et l'automatisation de certains aspects de cette gestion semble être un moyen pour gérer cet accroissement du trafic tout en gardant comme invariant un niveau de sécurité constant. Toutefois, cette augmentation du trafic pourrait entraîner un accroissement de la variabilité de la performance de l'ensemble des moyens de gestion du trafic aérien, en particulier dans le cas de dégradation de cette automatisation. Les systèmes de gestion du trafic aérien sont considérés comme complexes car ils impliquent de nombreuses interactions entre humains et systèmes, et peuvent être profondément influencés par les aspects environnementaux (météorologie, organisation, stress ...) et tombent, de fait, dans la catégorie des Systèmes Sociotechniques (STS) (Emery & Trist, 1960). A cause de leur complexité, les interactions entre les différents éléments (humains, systèmes et organisations) de ces STS peuvent être linéaires et partiellement non linéaires, ce qui rend l'évolution de leur performance difficilement prévisible. Au sein de ces STS, les systèmes interactifs doivent être utilisables, i.e. permettre à leurs utilisateurs d'accomplir leurs tâches de manière efficace et efficiente. Un STS doit aussi être résilient aux perturbations telles que les défaillances logicielles et matérielles, les potentielles dégradations de l'automatisation ou les problèmes d'interaction entre les systèmes et leurs opérateurs. Ces problèmes peuvent affecter plusieurs aspects des systèmes sociotechniques comme les ressources, le temps d'exécution d'une tâche, la capacité à d'adaptation à l'environnement... Afin de pouvoir analyser l'impact de ces perturbations et d'évaluer la variabilité de la performance d'un STS, des techniques et méthodes dédiées sont requises. Elles doivent fournir un support à la modélisation et à l'analyse systématique de l'utilisabilité et de la résilience de systèmes interactifs aux comportements partiellement autonomes. Elles doivent aussi permettre de décrire et de structurer un grand nombre d'informations, ainsi que de traiter la variabilité de chaque élément du STS et la variabilité liée à leurs interrelations. Les techniques et méthodes existantes ne permettent actuellement ni de modéliser un STS dans son ensemble, ni d'en analyser les propriétés d'utilisabilité et de résilience (ou alors se focalisent sur un sous-ensemble du STS perdant, de fait, la vision systémique). Enfin, elles ne fournissent pas les moyens d'analyser la migration de tâches suite à l'introduction d'une nouvelle technologie ou d'analyser la variabilité de la performance en cas de dégradation de fonctions récemment automatisées. Ces arguments sont développés dans la thèse et appuyés par une analyse détaillée des techniques de modélisation existantes et des méthodes qui leurs sont associées. La contribution présentée est basée sur l'identification d'un ensemble d'exigences requises pour pouvoir modéliser et analyser chacun des éléments d'un STS. Certaines de ces exigences ont été remplies grâce à l'utilisation de techniques de modélisation existantes, les autres grâce à l'extension et au raffinement d'autres techniques. Cette thèse propose une approche qui intègre 3 techniques en particulier : FRAM (centrée sur les fonctions organisationnelles), HAMSTERS (centrée les objectifs et activités humaines) et ICO (dédiée à la modélisation du comportement des systèmes interactifs). Cette approche est illustrée par un exemple mettant en œuvre les extensions proposées et l'intégration des modèles. Une étude de cas plus complexe sur la gestion du trafic aérien (changement de route d'un avion en cas de mauvaises conditions météorologiques) est ensuite présentée pour montrer le passage à l'échelle de l'approche. Elle met en avant les bénéfices de l'intégration des modèles pour la prise en compte de la variabilité de la performance des différents éléments d'un STSThe current European Air Traffic Management (ATM) System needs to be improved for coping with the growth in air traffic forecasted for next years. It has been broadly recognised that the future ATM capacity and safety objectives can only be achieved by an intense enhancement of integrated automation support. However, increase of automation might come along with an increase of performance variability of the whole ATM System especially in case of automation degradation. ATM systems are considered complex as they encompass interactions involving humans and machines deeply influenced by environmental aspects (i.e. weather, organizational structure) making them belong to the class of Socio-Technical Systems (STS) (Emery & Trist, 1960). Due to this complexity, the interactions between the STS elements (human, system and organisational) can be partly linear and partly non-linear making its performance evolution complex and hardly predictable. Within such STS, interactive systems have to be usable i.e. enabling users to perform their tasks efficiently and effectively while ensuring a certain level of operator satisfaction. Besides, the STS has to be resilient to adverse events including potential automation degradation issues but also interaction problems between their interactive systems and the operators. These issues may affect several STS aspects such as resources, time in tasks performance, ability to adjust to environment, etc. In order to be able to analyse the impact of these perturbations and to assess the potential performance variability of a STS, dedicated techniques and methods are required. These techniques and methods have to provide support for modelling and analysing in a systematic way usability and resilience of interactive systems featuring partly autonomous behaviours. They also have to provide support for describing and structuring a large amount of information and to be able to address the variability of each of STS elements as well as the variability related to their interrelations. Current techniques, methods and processes do not enable to model a STS as a whole and to analyse both usability and resilience properties. Also, they do not embed all the elements that are required to describe and analyse each part of the STS (such as knowledge of different types which is needed by a user for accomplishing tasks or for interacting with dedicated technologies). Lastly, they do not provide means for analysing task migrations when a new technology is introduced or for analysing performance variability in case of degradation of the newly introduced automation. Such statements are argued in this thesis by a detailed analysis of existing modelling techniques and associated methods highlighting their advantages and limitations. This thesis proposes a multi-models based approach for the modelling and the analysis of partly-autonomous interactive systems for assessing their resilience and usability. The contribution is based on the identification of a set of requirements needed being able to model and analyse each of the STS elements. Some of these requirements were met by existing modelling techniques, others were reachable by extending and refining existing ones. This thesis proposes an approach which integrates 3 modelling techniques: FRAM (focused on organisational functions), HAMSTERS (centred on human goals and activities) and ICO (dedicated to the modelling of interactive systems). The principles of the multi-models approach is illustrated on an example for carefully showing the extensions proposed to the selected modelling techniques and how they integrate together. A more complex case study from the ATM World is then presented to demonstrate the scalability of the approach. This case study, dealing with aircraft route change due to bad weather conditions, highlights the ability of the integration of models to cope with performance variability of the various parts of the ST

Modelling of Automation Degradation : a Case Study

National audienceThis paper presents a modeling approach that has been developed within the SPAD project for analyzing consequences of automation degradation in large socio-technical systems. This modeling approach involves two different notations: FRAM [6] and HAMSTERS [2], [8]. In previous work [7] we have proposed a synergistic approach integrating these two views for describing the evolution of system performances under automation degradation. The focus of the paper is on how the outcome of the models can be integrated to analyse system behavior. After describing the principles of such integration we exemplify it by using a standalone ATM simulator, and analyzing the possible degradations of a system for managing unmanned aircraft (RPAS)

Using Complementary Models-Based Approaches for Representing and Analysing ATM Systems' Variability

International audienceLarge-Scale Socio-Technical Systems, such as Air Traffic Management (ATM), are organizations where different interconnected systems work together to achieve a common goal. Analysis of variability is particularly challenging in these systems of systems due to the non-linear and complex interactions among social and technical functions. This paper proposes a systematic approach able to represent and to reason about the variability of such socio-technical systems. The proposed approach is based on the synergistic use of 3 models able to represent the variability from different points of view. This federation of models focusses the analysis on the relevant aspects of the systems of systems at different levels of granularity. The models taken into account for the representation of system variability are FRAM [12] focusing on organizational functions, HAMSTERS [17], which is centred on human goals and activities and ICO [20] which is dedicated to the representation of systems' behaviour (including the user interface). The paper presents a detailed development process describing how the models are built and analysed. This process is exemplified on a case study involving the AMAN (Arrival MANager) system

System Performance under Automation Degradation (WP-E project SPAD)

Available on: http://www.sesarinnovationdays.eu/papersInternational audienceIncreased automation is one of the main changes foreseen by SESAR in ATM. This will pose new challenges including possible automation degradation. The premise for the SPAD project is that degradation of systems automation is unavoidable due either to internal (e.g. human, software or system failure) or external (e.g. weather, strikes, malicious behaviors) events (or both). There is thus a need to understand, monitor and manage how automation degradation of a single system may propagate to the overall ATM system, and to define ways to confine and absorb degradation problems, with and without human contribution. There is also a need to estimate the implications of degradations for the overall ATM system performances. These aspects will be investigated by SPAD, which has the following aims: 1) understanding, modelling and estimating the propagation of automation degradation in ATM; 2) estimating the consequences of automation degradation on ATM performances; 3) supporting an effective intervention for the containment of automation degradation. This paper presents the early findings by the SPAD project after 6 months of work and presents the investigations that will be carried out in the next months

A framework for modeling the consequences of the propagation of automation degradation: application to air traffic control systems

International audienceThis paper presents a modelling approach for representing consequences of automation degradation in the context of a socio-technical network. This modelling approach involves two different notations. In previous work we have proposed a synergistic approach integrating these two views for describing the evolution of system performances under automation degradation. In the current paper we propose a more global approach encompassing the previous contribution and being specifically addressing the representation of consequences flowing from the occurrence of automation degradation. In this approach, four modelling levels of consequences are studied: direct consequences of automation propagation, consequences on the capacity to respond, consequences on resilience capacity and consequences on network performance. This stepwise refinement aims at acquiring and modelling additional information needed for being able to assess the consequences of automation degradation. The approach is exemplified on a case study in the domain of Air Traffic Management and more precisely Terminal Manoeuvre Area including an Arrival Manager (AMAN). Due to space constraints only the first level is fully presented

Une approche à base de modèles pour la modélisation et l'analyse de systèmes partiellement autonomes utilisables et résilients

La croissance prévisionnelle du trafic aérien est telle que les moyens de gestion actuels doivent évoluer et être améliorés et l'automatisation de certains aspects de cette gestion semble être un moyen pour gérer cet accroissement du trafic tout en gardant comme invariant un niveau de sécurité constant. Toutefois, cette augmentation du trafic pourrait entraîner un accroissement de la variabilité de la performance de l'ensemble des moyens de gestion du trafic aérien, en particulier dans le cas de dégradation de cette automatisation. Les systèmes de gestion du trafic aérien sont considérés comme complexes car ils impliquent de nombreuses interactions entre humains et systèmes, et peuvent être profondément influencés par les aspects environnementaux (météorologie, organisation, stress ...) et tombent, de fait, dans la catégorie des Systèmes Sociotechniques (STS) (Emery & Trist, 1960). A cause de leur complexité, les interactions entre les différents éléments (humains, systèmes et organisations) de ces STS peuvent être linéaires et partiellement non linéaires, ce qui rend l'évolution de leur performance difficilement prévisible. Au sein de ces STS, les systèmes interactifs doivent être utilisables, i.e. permettre à leurs utilisateurs d'accomplir leurs tâches de manière efficace et efficiente. Un STS doit aussi être résilient aux perturbations telles que les défaillances logicielles et matérielles, les potentielles dégradations de l'automatisation ou les problèmes d'interaction entre les systèmes et leurs opérateurs. Ces problèmes peuvent affecter plusieurs aspects des systèmes sociotechniques comme les ressources, le temps d'exécution d'une tâche, la capacité à d'adaptation à l'environnement... Afin de pouvoir analyser l'impact de ces perturbations et d'évaluer la variabilité de la performance d'un STS, des techniques et méthodes dédiées sont requises. Elles doivent fournir un support à la modélisation et à l'analyse systématique de l'utilisabilité et de la résilience de systèmes interactifs aux comportements partiellement autonomes. Elles doivent aussi permettre de décrire et de structurer un grand nombre d'informations, ainsi que de traiter la variabilité de chaque élément du STS et la variabilité liée à leurs interrelations. Les techniques et méthodes existantes ne permettent actuellement ni de modéliser un STS dans son ensemble, ni d'en analyser les propriétés d'utilisabilité et de résilience (ou alors se focalisent sur un sous-ensemble du STS perdant, de fait, la vision systémique).The current European Air Traffic Management (ATM) System needs to be improved for coping with the growth in air traffic forecasted for next years. It has been broadly recognised that the future ATM capacity and safety objectives can only be achieved by an intense enhancement of integrated automation support. However, increase of automation might come along with an increase of performance variability of the whole ATM System especially in case of automation degradation. ATM systems are considered complex as they encompass interactions involving humans and machines deeply influenced by environmental aspects (i.e. weather, organizational structure) making them belong to the class of Socio-Technical Systems (STS) (Emery & Trist, 1960). Due to this complexity, the interactions between the STS elements (human, system and organisational) can be partly linear and partly non-linear making its performance evolution complex and hardly predictable. Within such STS, interactive systems have to be usable i.e. enabling users to perform their tasks efficiently and effectively while ensuring a certain level of operator satisfaction. Besides, the STS has to be resilient to adverse events including potential automation degradation issues but also interaction problems between their interactive systems and the operators. These issues may affect several STS aspects such as resources, time in tasks performance, ability to adjust to environment, etc. In order to be able to analyse the impact of these perturbations and to assess the potential performance variability of a STS, dedicated techniques and methods are required. These techniques and methods have to provide support for modelling and analysing in a systematic way usability and resilience of interactive systems featuring partly autonomous behaviours. They also have to provide support for describing and structuring a large amount of information and to be able to address the variability of each of STS elements as well as the variability related to their interrelations. Current techniques, methods and processes do not enable to model a STS as a whole and to analyse both usability and resilience properties. Also, they do not embed all the elements that are required to describe and analyse each part of the STS (such as knowledge of different types which is needed by a user for accomplishing tasks or for interacting with dedicated technologies). Lastly, they do not provide means for analysing task migrations when a new technology is introduced or for analysing performance variability in case of degradation of the newly introduced automation. Such statements are argued in this thesis by a detailed analysis of existing modelling techniques and associated methods highlighting their advantages and limitations. This thesis proposes a multi-models based approach for the modelling and the analysis of partly-autonomous interactive systems for assessing their resilience and usability. The contribution is based on the identification of a set of requirements needed being able to model and analyse each of the STS elements. Some of these requirements were met by existing modelling techniques, others were reachable by extending and refining existing ones. This thesis proposes an approach which integrates 3 modelling techniques: FRAM (focused on organisational functions), HAMSTERS (centred on human goals and activities) and ICO (dedicated to the modelling of interactive systems). The principles of the multi-models approach is illustrated on an example for carefully showing the extensions proposed to the selected modelling techniques and how they integrate together. A more complex case study from the ATM World is then presented to demonstrate the scalability of the approach. This case study, dealing with aircraft route change due to bad weather conditions, highlights the ability of the integration of models to cope with performance variability of the various parts of the ST

Some Issues with Interaction Design and Implementation in the Context of Autonomous Interactive Critical Systems

Workshop session of ACM International Conference on Human Factors in Computer Science, CHI 2012International audienceIncreasing analysis and decision capabilities in systems is a classical approach followed by designers and engineers to support operators in the command and control tasks of more and more complex systems. However, designing interfaces that will be used to operate intelligent systems and (partly) autonomous systems is a very complex activity altering in depth the development process of these systems. In the early days, the development of such systems was rather limited and thus easier to manage even though bad designs have been widely and frequently reported (e.g. [13, 15]). Nowadays, they can be integrated to several critical systems, such as Air Traffic Control management systems, as illustrated by the example presented in the case study. This position paper raises some issues related to the design and implementation of safety-critical user interfaces featuring intelligent and (partly) autonomous behaviors

Extending Procedural Task Models by Explicit and Systematic Integration of Objects, Knowledge and Information

Task analysis can be considered as a fundamental component of user centered design methods as it provides a unique way of analyzing in a systematic way users' roles and activities. A widely used way of storing the information gathered during that phase in a structured and exhaustive way is to build task models which are then amenable to verification of properties or to performance evaluation. In widely used notations such as Hierarchical Task Analysis (HTA) or CTT (Concur Task Tree), information or objects manipulated by the users while performing the tasks does not receive a similar treatment as the sequencing of tasks which is usually carefully and exhaustively described. This paper proposes a systematic account for the various concepts manipulated by the users while performing tasks. Such concepts include different types of knowledge (declarative, situational, procedural and strategic), objects (manipulated by the user) and information. These concepts are systematically represented in a set of extensions of the HAMSTERS notation allowing the analysis of concepts-related properties such as learning curve, complexity, information workload,... We demonstrate the application of the approach on the example of a two players game making explicit the connection between these extended task models and the user interface of the game

Task-Model Based Assessment of Automation Levels: Application to Space Ground Segments

International audienceDesigning systems in such a way that as much functions as possible are automated has been the driving direction of research and engineering in aviation, space and more generally in computer science for many years. In the 90's many studies (e.g. [12] related to the notion of mode confusion) have demonstrated that fully automated systems are out of the grasp of current technologies and that additionally migrating functions [2] from the operator to the system might have disastrous impact on operations both in terms of safety and usability. In order to be able to design automation with a hedonic view of the involved factors (safety, usability, reliability, ...) a complete understanding of operator's tasks is required prior to considering migrating them to the system side. This paper proposes a contribution for reasoning about automation designs using a model-based approach exploiting refined task models. These models describe operations with enough details in order to reason about automation and to rationalize automation designs. In this paper we present how such representations can support the assessment of alternative design options for automation. The proposed approach is applied to satellite ground segments

Model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. This generic model-based approach is exemplified on a safety critical system from the space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new rocedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus + context way improving usability increasing efficiency and effectiveness